Efficient Computation of Time-Bounded Reachability Probabilities in Uniform Continuous-Time Markov Decision Processes
نویسندگان
چکیده
A continuous-time Markov decision process (CTMDP) is a generalization of a continuous-time Markov chain in which both probabilistic and nondeterministic choices co-exist. This paper presents an efficient algorithm to compute the maximum (or minimum) probability to reach a set of goal states within a given time bound in a uniform CTMDP, i.e., a CTMDP in which the delay time distribution per state visit is the same for all states. We prove that these probabilities coincide for (time-abstract) history-dependent and Markovian schedulers that resolve nondeterminism either deterministically or in a randomized way.
منابع مشابه
Analysis and scheduler synthesis of time - bounded reachability in continuous - time Markov decision processes
Continuous-time Markov decision processes (CTMDPs) are stochastic models in which probabilistic and nondeterministic choices co-exist. Lately, a discretization technique has been developed to compute time-bounded reachability probabilities in locally uniform CTMDPs, i.e. CTMDPs with state-wise constant sojourn-times. We extend the underlying value iteration algorithm, such that it computes an -...
متن کاملVerification of continuous-space stochastic systems
This thesis deals with verification algorithms for inhomogeneous continuous time Markov chains (ICTMC), discrete time stochastic hybrid systems (DTSHS) and Markovian timed automata (MTA). For all of these three models we define the notions of time-bounded and time-unbounded reachability. We use time-bounded and time-unbounded reachability in order to compute the satisfiability probability of an...
متن کاملLTL Model Checking of Time-Inhomogeneous Markov Chains
We investigate the problem of verifying linear-time properties against inhomogeneous continuous-time Markov chains (ICTMCs). A fundamental question we address is how to compute reachability probabilities. We consider two variants: time-bounded and unbounded reachability. It turns out that both can be characterized as the least solution of a system of integral equations. We show that for the tim...
متن کاملTime-Bounded Reachability in Continuous-Time Markov Decision Processes
This paper solves the problem of computing the maximum and minimum probability to reach a set of goal states within a given time bound for locally uniform continuous-time Markov decision processes (CTMDPs). As this model allows for nondeterministic choices between exponentially delayed transitions, we define total time positional (TTP) schedulers which rely on the CTMDP’s current state and the ...
متن کاملMaximal Cost-Bounded Reachability Probability on Continuous-Time Markov Decision Processes
In this paper, we consider multi-dimensional maximal cost-bounded reachability probability over continuous-time Markov decision processes (CTMDPs). Our major contributions are as follows. Firstly, we derive an integral characterization which states that the maximal cost-bounded reachability probability function is the least fixed-point of a system of integral equations. Secondly, we prove that ...
متن کامل